[8.0] feat: PoolCE takes care of RAM requirements#8232
Conversation
DIRACGridBot
added
the
alsoTargeting:integration
| if availableMemory < requestedMemory: | ||
| return None | ||
|
|
||
| return requestedMemory |
There was a problem hiding this comment.
I might have overlooked something, do we specify a default RAM value for the jobs?
Because if I understand correctly, getRAMInUse() depends on the value returned by _getMemoryForJobs(), which would be 0 if the tag *GB or *GB_MAX is not specified (?).
So if we have n jobs with no "RAM" tag, we will always have getRAMInUse = 0.
There was a problem hiding this comment.
There's no obvious default value for RAM (for processors we specify 1 which is a slightly more obvious default value). So, yes, as it stands the default value for RAM is 0.
There was a problem hiding this comment.
RAM and NumberOfProcessors should be treated in exactly the same way. These are resources with integer values, having default values and summing up for jobs on the same node. So, may be common methods (or class) can be introduced for those, may be other tags can be added eventually, e.g. disk space.
There was a problem hiding this comment.
Having RAM in GBs isn't fine enough for an integer value, having less than 1GB per core is sensible on some systems and jobs.
There was a problem hiding this comment.
Could this be made MB instead?
sfayer
left a comment
sfayer
left a comment
There was a problem hiding this comment.
Hi,
I've done a quick review: Please let me know if you need anything else from me on this. (I didn't quite hear what you said about it in the most recent BiLD meeting: there was some local noise just as you were talking about it).
The other CEs (singularity/inprocess) should probably set the same memory setting based on the tags too if they've not already been set by PoolCE for cases where the PoolCE isn't in use.
Regards,
Simon
fstagni
force-pushed
the
80_Pool_RAM
branch
3 times, most recently
from
6c09d92 to
ee00569
Compare
fstagni
force-pushed
the
80_Pool_RAM
branch
2 times, most recently
from
2db1187 to
608d57f
Compare
fstagni
force-pushed
the
80_Pool_RAM
branch
3 times, most recently
from
2883631 to
f2e8f9e
Compare
fstagni
force-pushed
the
80_Pool_RAM
branch
3 times, most recently
from
b8b1635 to
e41e706
Compare
fstagni
force-pushed
the
80_Pool_RAM
branch
3 times, most recently
from
f859666 to
3ce23be
Compare
|
This PR is replaced by #8366 which targets v9.0. As said, these are a bit too many changes for 8.0 branch. |
7 participants
BEGINRELEASENOTES
*WMS
NEW: InProcessComputingElement takes RAM requirements into consideration for jobs' matching
NEW: PoolComputingElement takes RAM requirements into consideration when subdividing the pool
NEW: SingularityComputingElement can enforce CG2 RAM limits
ENDRELEASENOTES